Modern data centers house thousands of servers, and each server typically includes two or more heat-generating microprocessors. Each microprocessor can easily produce more than 40 thermal watts per square centimeter, and future microprocessors are expected to produce even higher heat fluxes as semiconductor technology continues to progress. It follows that the total amount of heat generated by servers in a data center is substantial. Unfortunately, removing heat from the data center using conventional systems is costly and inefficient. For example, removing heat by air conditioning requires significant capital expenditures on large air conditioning units as well as significant ongoing operating expenditures to power the air conditioning units. The units suffer from poor thermodynamic efficiency, which translates to high utility bills for data center operators. To reduce the cost of operating data centers, and thereby reduce the cost of cloud storage services that rely on data centers, there is a strong need to cool servers more efficiently.
According to the U.S. Department of Energy, nearly three percent of all electricity used in the United States is devoted to powering data centers and computer facilities. Approximately half of this electricity goes toward power conditioning and cooling. Increasing the efficiency of cooling systems for data centers and computer facilities would lead to dramatic savings in energy nationwide. More efficient cooling systems are also needed in transportation systems due to increasing adoption of hybrid and electric vehicles that rely on complex electrical components, including batteries, inverters, and electric motors, that produce significant amounts of heat that must be dissipated. Cooling systems capable of more efficiently cooling these electrical components would translate to an increase in range and utility for these vehicles.
The majority of computer systems in residential and commercial settings are cooled using forced air cooling systems in which room air is forced, by one or more fans, over finned heat sinks mounted on microprocessors, power supplies, or other electronic devices. The heat sinks add mass and cost to the computers and place mechanical stress on the electronic components to which they are mounted. If a computer is subject to vibration, such as vibration caused by a fan, a heat sink mounted on top of a microprocessor can oscillate in response to the vibration and can fatigue the electrical connections that attach the microprocessor to the motherboard of the computer.
Another downside of air cooling systems is that cooling fans commonly operate at high speeds and can be quite noisy. As air passes over electronic devices, the air, which is at a lower temperature than the surfaces of the electronic devices, absorbs heat from the electronic devices, thereby cooling the devices. These air cooling systems are inherently limited in terms of performance and efficiency due to the low volumetric heat capacity of air, which is much lower than the volumetric heat capacity of water and other coolants. Because air has such a low heat capacity, high flow rates are required to ensure adequate cooling of even relatively small heat loads. For instance, a flow rate of about 5 to 10 cubic feet per minute (cfm) of air is needed to cool a 100-watt heat load. For heat sources such as microprocessors, which as mentioned above can easily produce more than 40 watts per square centimeter, very high volumetric flow rates are required to prevent overheating. For an installation of one rack of servers, which is commonly used in computer rooms at small businesses and schools, two air conditioning units sized for a typical U.S. home are required to cool the computer room. Typical data centers, which can have several hundred racks of servers, must be equipped with special computer room air conditioning (“CRAC”) units that are large and expensive and must be professionally installed, often requiring substantial modifications to the facility to accommodate the CRAC, including installation of structural supports, custom air ducting, and electrical wiring.
Many electronic devices operate less efficiently as their temperature increases. As one example, a typical microprocessor operates less efficiently as its junction temperature increases. FIG. 64 shows a plot of power consumption in watts versus junction temperature. The bottom curve shows static power consumption of a microprocessor and the top curves show total power consumption for switching speeds of 1.6 GHz and 2.4 GHz, respectively. Total power consumption includes both static power consumption and dynamic power consumption, which varies with switching frequency. As shown in FIG. 64, as the temperature of the microprocessor increases, it consumes more power to provide the same performance. In air cooling systems, it is common for fully utilized microprocessors to operate at or near their maximum rated temperature, resulting in poor operating efficiency. In the example shown in FIG. 64, the microprocessor uses over 35% more power when operating at 95 degrees C. than when operating at 45 degrees C. To conserve energy, it is therefore desirable to provide a cooling system that will allow the microprocessor to operate consistently at lower temperatures. Providing a consistently lower operating temperature for the microprocessor can also extend its useful life and can avoid unnecessary downtime of the computer due to an unsafe junction temperature.
Operating speeds of next generation microprocessors will continue to increase, as will heat fluxes (defined as heat load per unit area) produced by those next generation microprocessors. Conventional air cooling systems will soon be incapable of efficiently and effectively cooling these next generation microprocessors. To effectively cool next generation microprocessors, it is desirable to provide a cooling system that is significantly more effective and efficient than existing air cooling systems and is capable of managing high heat fluxes that will be produced by next generation microprocessors.
Pumped liquid cooling systems can provide improved thermal performance over conventional air cooling systems. Pumped liquid cooling systems typically include the following items connected by tubing: a heat sink attached to the microprocessor, a liquid-to-air heat exchanger, and a pump. A liquid coolant is circulated through the system by the pump. As the liquid coolant passes through channels in the heat sink, heat from the microprocessor is transferred through the thermally conductive heat sink to the coolant, thereby increasing the temperature of the coolant and transferring heat away from the microprocessor. The heat sink is typically designed to maximize heat transfer by maximizing the surface area of the channels through which the liquid passes. For example, the heat sink can be a micro-channel heat sink that utilizes fine fin channels through which the liquid coolant flows. The heated liquid coolant exiting the heat sink is then circulated through a liquid-to-air heat exchanger to reduce the temperature of the liquid coolant before it is circulated back to the pump for another cycle.
Use of closed liquid cooling systems is beginning to migrate from high performance computers to personal computers. Unfortunately, liquid cooling systems have performance constraints that will prevent them from effectively cooling next generation microprocessors. This is because liquid cooling systems rely solely on transferring sensible heat by increasing the temperature of a liquid coolant as it passes through a heat sink. The amount of heat that can be transferred is a function of, among other factors, the thermal conductivity of the fluid and the flow rate of the fluid. Dielectric fluids do not have sufficient thermal conductivities to be used in liquid cooling systems. Instead, water or a water-glycol mixture is commonly used due its significantly higher thermal conductivity. Unfortunately, if a leak develops in a liquid cooling system that uses water, the water can destroy the server and potentially an entire rack of servers. With the price of a single server being thousands of dollars, many data center operators are simply unable to accept this risk of loss.
While more effective than air cooling, transferring heat by sensible heating requires significant flow rates of liquid coolant, and achieving high flow rates often necessitates high fluid pressures. Consequently, a liquid cooling system designed to cool a modern microprocessor can require a large pump, or a series of small pumps positioned throughout the liquid cooling system, to ensure an adequate liquid coolant pressure and flow rate. Operating large pumps, or a series of small pumps, uses a significant amount of energy and diminishes the efficiency of the liquid cooling system.
Although liquid cooling systems have proven adequate at cooling modern microprocessors, they will be unable to adequately cool next generation microprocessors while maintaining practical physical dimensions and specifications. For instance, to cool a next generation microprocessor, liquid cooling systems will require very high flow rates (e.g. of water), which will require large, heavy duty cooling lines (e.g. greater than ¾″ outer diameter), such as rigid copper tubing or reinforced rubber cooling lines, that will be difficult to route in any practical manner in and out of a server housing. If installed in a server, these large plumbing lines will block access to electrical components within the server, thereby frustrating maintenance of the server. These large plumbing lines will also prevent drawers on a server rack from opening and closing as intended, thereby preventing the server from being easily accessed and further frustrating maintenance of the server. As mentioned above, water poses a catastrophic risk to servers, and increasing the pressure and flow rates of water in and out of servers only increases this risk. Consequently, increasing the capabilities of existing liquid cooling systems to meet the cooling requirements of next generation microprocessors is simply not a practical or viable option. Without further innovation in the area of cooling systems, the development of next-generation microprocessors will be hampered.
As noted above, liquid cooling systems commonly rely on flowing liquid water through channels in finned heat sinks. The heat sinks are often indirectly coupled to a heat source via a metal base plate, thermal paste, such as solder thermal interface material (STIM) or polymer thermal interface material (PTIM), and/or a direct bond adhesive. While this approach can be more effective than air cooling, the intervening materials between the water and the heat source (e.g. the microprocessor) induce significant thermal resistance, which reduces the overall efficiency of the cooling system. The intervening materials also add cost and time to manufacturing and installation processes, constitute additional points of failure, and create potential disposal issues. Finally, the intervening materials render the system unable to adapt to local hot spots on a heat source. Consequently, the liquid cooling system must be designed to accommodate the maximum anticipated heat load of one or more localized hot spots on the surface of the heat source (e.g. to adequately cool one hot core of a multicore processor), resulting in additional cost and complexity of the entire liquid cooling system.
Unlike water, dielectric coolants can be placed in direct contact with electronic devices and not harm them. Unfortunately, some dielectric coolants have a lower heat capacity than water, so they are not well suited for use in pumped liquid cooling systems, since they may be unable to adequately cool a microprocessor. Presently, use of dielectric coolants requires more aggressive cooling techniques, such as immersion cooling, to achieve a desired level of heat transfer.
Immersion cooling is an aggressive form of liquid cooling where an entire electronic device (e.g. a server) is submerged in a vat of dielectric coolant (e.g. HFE-7000 or mineral oil). Unfortunately, immersion cooling vats are large, costly, and heavy, especially when filled with a dielectric coolant. Existing vats hold upwards of 250 gallons of coolant can weigh more than 8,000 pounds when full of coolant. Typically, a room must be specially engineered to accommodate the immersion cooling vat, and containment systems may need to be specially designed and installed in the room as a precaution against vat failure. When using 250 gallons of coolant, the cost of the coolant becomes a significant capital expenditure. Certain coolants, such as mineral oil, can act as solvents and, over time, remove certain identifying information from a motherboard. For example, product labels (e.g. stickers containing serial numbers and bar codes) and other markings (e.g. values on capacitors and other devices) are prone to wash off in the vat, due to a continuous flow of the coolant over all portions of the electronic device. Over time, the coolant can become contaminated and may need to be replaced, resulting in an additional expense and downtime.
Another cooling approach, known as spray cooling or spray evaporative cooling, uses atomized sprays. In this approach, atomized liquid coolant is sprayed directly on a surface through air or vapor. As a result, small droplets impinge on the heated surface forming a thin film of liquid directly on a heated surface. Heat is then transferred from the heated surface to the liquid either by sensible heating of the bulk liquid or by boiling off of a fraction of the liquid (i.e. latent heating). This is a very efficient method of removing high heat fluxes from small surfaces. Unfortunately, the margin for error in spray cooling systems is very narrow and the onset of dry out and critical heat flux is a constant concern that can have catastrophic consequences. Critical heat flux is a condition where evaporation of coolant from the surface to be cooled prevents atomized liquid from reaching and cooling the surface, often resulting in run-away device temperatures and rapid failure. Great care must be taken to ensure uniform coverage of the spray on the heated surface and adequate drainage of fluid from the heated surface. Although achievable in a laboratory setting, mainstream adoption of spray cooling has been hampered by several factors. First, spray cooling requires a significant working volume to enable atomized sprays to form, which results in non-compact cooling components, making it impractical for most consumer products. Second, atomizing the liquid requires a significant amount of pressure upstream of the atomizer to generate an appropriate pressure drop at the atomizer-air interface to enable atomized sprays to form. Maintaining this amount of pressure within the system consumes a significant amount of energy. Third, high flow rates of atomized sprays are required to prevent dry out or critical heat flux from occurring. In the end, it has proven difficult to design a practical and compact spray cooling system, despite a large amount of time and effort that has been expended to do so.
In view of the foregoing discussion, efficient, scalable, high-performing methods and apparatuses are needed for cooling devices, such as microprocessors and power electronics that produce high heat fluxes.